Generally, for efficient video compression, a video encoder removes redundancy and performs motion estimation in units of macroblocks (MBs) between video frames. Accordingly, since encoding is performed with reference to a redundant portion in an area encoded before, a data size may be reduced. However, a case where a corresponding area is lossy in a video transmission process affects not only lossy frames but also a plurality of frames behind them that reference the lossy area.
In addition, in order to reduce such an effect in various channels, error resilience coding is necessary, and a random intra refresh (RIR) scheme, which is strongest error resilience coding scheme, is widely used.
However, for a video codec based on MRF such as H.264/AVC that had an MB refresh and is useable according to the RIR scheme, a phenomenon that an error resilience effect is decreased occurs, which is discussed in “M. G. S. Moiron, I. Ali and M. Fleury, “Limitations of multiple reference frames with cyclic intra-refresh line for h.264/avc,” ELECTRONIC LETTERS, vol. 47, pp. 103104, January 2011”, wherein the H.264 is a kind of H.26x video standard proposed ITU-T and is normally called H.264/AVC or AVC/H.264, H.264/MPEG-4 AVC, MPEG-4/H.264 AVC and called AVC, which in MPEG technology is a name of MPECG-4 part 10.
As described above, the error weakness phenomenon in the codec using the MRF is additionally described below with a schematic description using H.264/AVC.
H.264/AVC is a video codec widely used nowadays and such a H.264/AVC standard is formed of two layers divided into a video coding layer (VCL) and a network abstraction layer (NAL).
The VCL supports technique elements pertaining to a video encoding process, and the NAL is between with a lower layer system transmitting and storing encoded information. Encoded data generated in the VCL is packetized in a real-time transport protocol (RTP) in units of NALs in a network environment, and received RTP packets are recovered to a video by a decoder in a reception stage. The RTP packets may be at risk of loss in an environment such as a wireless network in which channel errors may occur, and in this case, the decoder in the reception stage may not wholly or perfectly recover the video.
The H.264/AVC encoder supports error resilience schemes for performing a preprocess in order to reduce deterioration in display quality of an image, which occurs at the time of recovering the video through the decoder in the reception stage in a state where some data is lossy due to channel errors, and the decoder supports error concealment schemes for performing a post-process in order to continuously recover the video when a portion of data is lossy due to the errors.
In addition, an H.264/AVC standard decoder performs motion estimation by using MRF to improve compression efficiency. When a frame Ft at time t is encoded, since an optimal block may be selected from arbitrary frames between Ft−n and Ft+n as well as a previous frame Ft−1, high efficiency motion estimation may be realized. This results from the fact that a background covered by a moving object and thus not shown, may be estimated from a frame at another time by using the MRF.
In addition, like this, a video encoded by using the motion estimation is recovered with reference to video data from reference frames through motion compensation in the decoder.
Here, when a portion of video data is lossy in an environment such as a wireless network where errors may occur, an error propagation phenomenon occurs affecting other frames on which motion compensation is performed with reference to a lossy area.
The H.264/AVC standard also supports the RIR scheme in order to prevent errors from being propagated like this. In other words, when each frame is encoded, some MBs are independently encoded without motion estimation performed thereon, and since, at the time of decoding, motion compensation is not performed on a corresponding area, the error propagation is prevented. In the case where one previous frame is used as a reference frame, although a corresponding frame is lossy, an intra refresh area of the frame behind the previous frame is normally recovered and the error may be rapidly recovered since subsequent frames refer to the normally recovered area.
However, when MRF are used, the error may not be rapidly recovered although periodic intra refresh data is inserted. This is because when loss occurs in the previous frame Ft−1, a portion of area of Ft will be recovered with intra refresh data, but, when motion compensation is performed from Ft−1 not from Ft at a subsequent frame Ft+1, the recovered area may be lossy again. Accordingly, when decoding is performed by using MRF, a compression efficiency increases, but when some data is lossy in a process of transmitting a video in a network environment, display quality of the image becomes more deteriorated.
Referring to FIGS. 1A and 1B, illustrated is a phenomenon appearing when some of compressed video data is damaged in a video codec using the MRF and RIR scheme, and a method proposed to correct this.
For the case of FIG. 1A, first, through a picture frame (P-frame) 11 for which the p-frame is called a first P-frame and subsequent P-frames are sequentially called a second to n-th P-frame for convenience, a specific inter-coded macroblock
(MB) 12b of the second P-frame 12 is image-corrected, and at this point, since the image correction is performed on the specific inter-coded MB included in the second P-frame 12 by an inter-coded MB 11b included a damaged area 11a of the first P-frame 11, a normal image correction is not performed.
In addition, like this, as the specific inter-coded MB 12b of the second P-frame 12, which is not normally corrected, is used for image correction of a specific inter-coded MB 14b included in the fourth P-frame 14, a corresponding coded MB 14b of the fourth P-frame 14 is not normally corrected. Furthermore, the third P-frame 13 exemplarily shows that image correction is performed on the third P-frame 13 within itself with the intra-coded MB 13b. 
In addition, for the case of FIG. 1B, a specific inter-coded MB 22b of the second P-frame is corrected through the first P-frame 21 having a damaged area 21a, and at this point, since the specific inter-coded MB 22b included in the second P-frame 22 is corrected with an inter-coded MB 21b included in the damaged area 21a of the first P-frame 21, normal image correction is not performed.
However, the third P-frame 23 is corrected within itself with the intra-coded MB 23b. The fourth P-frame 24 exemplarily shows that the intra-coded MB 23b of the third P-frame 23 is used for image correction on the specific inter-coded MB 24b of the fourth P-frame 24. In other words, since the intra-coded MB 23b is used for the image correction of the specific inter-coded MB 24b of the fourth P-frame 24, the image correction is performed with the error propagation blocked.